​In many applications, image segmentation is required at some point of the analysis, eg when assessing the texture of a tumor (either in PET or in MRI), when defining the gross tumor volume or the biological tumor volume for treatment planning in radiotherapy, or when computing kinetic parameters. In PET, segmentation is always challenging due to the limited spatial resolution in PET images (from 4 to 8 mm depending on the PET system - from scanner dedicated to brain research to whole-body clinical scanners). Guiding PET image segmentation by integrating higher resolution MR images (~1 mm for anatomical MRI), and conversely use PET information to guide MR segmentation, is thus appealing. A possible strategy consists in using an initial tumor localization derived from PET data, which is then used to constrain a deformable model for accurately segmenting the tumor in high resolution images, as done previously for TEP-CT data (see Figure). The latter scheme allows for many refinements such as performing tumor segmentation into parts by combining deformable models and fuzzy sets, or jointly exploiting the information contained in multi-protocol MRI scans using multi-phase and multi-channel level set techniques. The current limitations of these methods are currently twofold: first, most of them are relatively sensitive to the accuracy of image spatial registration between the images; second managing possible mismatches between modality-specific information provided by the two imaging modalities remains an issue. Several aspects will be investigated to move forward in that research. 

Since 2006, IMT-Telecom ParisTech has performed seminal work on structural knowledge modeling (such as fuzzy spatial relations embedded in a graph or ontological representation) with the aim of guiding segmentation and recognition of structures in images, based on the fact that in both normal and pathological contexts, spatial relations are much more stable than shape information that is highly prone to individual variability. Sequential and global constraint satisfaction schemes have been elaborated, yielding promising results on tumoral MR data. Building on these works, we will investigate the use of structural knowledge and spatial reasoning for modeling and understanding the structure and evolution of tumors within their anatomical context from PET-MRI data. Research will be carried out along two directions:

• performing tumor segmentation using novel deformable tumoral organ models incorporating weak constraints on tumor location;

• assessing the impact of a tumor on adjacent structures thanks to a description of image content jointly in qualitative and quantitative terms.

To comply with the foreseeable variety of PET-MRI protocols, which will likely include multiple MR acquisitions delivering data with potentially different dimensions (eg T2+functional MRI) and statistics (eg T2+DCE-MRI), a unified and versatile framework for measuring similarity/discrepancy in multimodal multi-channel contexts is required. IMT-Telecom SudParis has previously developed region-based segmentation schemes using information-theoretic similarity measures in the level set framework. Recently, novel high-dimensional geometric entropic estimators have been identified, paving the way for multi-feature multi-channel statistical segmentation schemes that will be developed in this project. 

For specific anatomical territories investigated via multi-protocol MRI (e.g. abdominal and thoracic regions undergoing cardio-respiratory motion), technological constraints are likely to limit the applicability of motion correction within reconstruction to a subset of MR data. Hence, MR-based retrospective registration will be needed for aligning the remaining MR images. In addition, multimodal nonrigid registration will remain a need in longitudinal PET-MRI studies and radiotherapy planning. Depending on the PET-MRI protocol, various alignment strategies will be investigated based on IMT previous experience:

• landmark-based: developments will rely on a nonlinear registration method developed by IMT-Telecom ParisTech, combining a breathing model with constraints on structure interfaces and tumors, which was shown to yield physiologically consistent results for PET-CT;

• voxel-based: relying on a versatile multimodal high-dimensional statistical registration model developed by IMT-Telecom SudParis.

An option to deal with the uncertainties associated with the real match between the structural or functional edges present in the images to be combined is to propagate the uncertainties associated to these edges in the subsequent quantification task so as to associate an uncertainty to the measured parameters. The CEA-SHFJ group will investigate this option, with the aim to take advantage of priors derived from MRI images to segment PET images while providing a standard deviation associated with the parameters derived from the PET region. 

Participants / Partners:

​

​

​

• Helene Urien, PhD candidate
• Isabelle Bloch, Professor at Telecom ParisTech, thesis advisor
• Nicolas Rougon, Associate Professor at Telecom SudParis, thesis co-advisor, task leader
• Irene Buvat, Senior Researcher at CEA-SHFJ, project coordinator, thesis co-advisor